RepliPRI: Challenges in Replicating Studies of Online Privacy

Presented at RepliCHI2013. Copyright c © 2013 for the individual papers by the papers authors. Copying permitted only for private and academic purposes. This volume is published and copyrighted by its editors. Abstract Replication of prior results has recently attracted attention and interest from the CHI community. This paper focuses on the challenges and issues faced in carrying out meaningful and valid replications of HCI studies. I attribute these challenges to two main underlying factors: (i) a domain of inquiry that simultaneously covers people, social systems, and technology; and (ii) deficiencies in result reporting and data archiving. Using examples from investigations of online privacy, I outline how these challenges manifest themselves in HCI studies. Longitudinal approaches, international collaboration, and sharing of study instruments could help address these challenges.